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1. INTRODUCTION 

Since the advent of meteorological satellites in the 
1960's, numerous experiments have been conducted in 
order to evaluate the impact of :hese and other data on 
atmospheric analysis and prediction. Such studies have 
included both OSE'S and OSSE's. The OSE’s were 
conducted to evaluate the impact of specific observa- 
tions or classes of observations on analyses and fore- 
casts. Such experiments have been performed for se- 
lected types of conventional da a and for various satel- 
lite data sets as they became available. (See for exam- 
ple the 1989 ECMWF/EUMET5AT workshop proceed- 
ings on "The use of satellite data in operational numer- 
ical weather prediction” and the references contained 
therein.) The OSSE's were conducted to evaluate the 
potential for future observing systems to improve Nu- 
merical Weather Prediction (NWP) and to plan for the 
Global Weather Experiment and more recently for EOS 
(Atlas et al, 1985a; Arnold and Dey, 1986; Hoffman 
et al., 1990). In addition, OSSE’s have been run to 
evaluate trade-offs in the design of observing systems 
and observing networks (Atlas and Emmitt, 1991; Ro- 
haly and Krishnamurti, 1993), ;md to test new method- 
ology for data assimilation (At as and Bloom, 1989). 


2. EXPERIMENTAL DESIGN 

Although there are many possibilities for how an 
OSE may be conducted, the rrost typical procedure is 
as follows: First a "Control'' data assimilation cycle is 
performed. This is followed by one or more experimen- 
tal assimilations in which a pai:icular type of data (or 
specific observations) are either withheld or added to 
the Control. Forecasts are then generated from both 
the Control and experimental assimilations every few 
days (to achieve relative independence of the forecast 
sample). The analyses and iorecasts from each as- 
similation are then verified and compared in order to 
determine the impact of each data type being evalu- 
ated. 
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Experiments performed in this manner provide a 
quantitative assessment of the value of a selected type 
of data to the specific data assimilation system (DAS) 
that was used. In addition, the OSE also provides use- 
ful information on the effectiveness of the DAS. This 
information can be used to improve the utilization of 
this and other data in the DAS, as well as to determine 
the value of the data. 

The methodology currently used for OSSE's is 
very similar to that described above for OSE’s. How- 
ever, this methodology has evolved considerably since 
these experiments were first carried out in the 1950’s 
and 60's (Arnold and Dey, 1986). The earliest simu- 
lation studies proceeded according to the following se- 
quence of steps. First, an artificial history of the atmo- 
sphere wascreated by numerical integration of a model. 
Second, simulated "data” were created from the his- 
tory by the addition of random variations to the history 
values for temperature, wind, and pressure. Third, the 
numerical integration that created the history was re- 
peated, but with the meteorological variables in the 
model replaced by the simulated data at locations and 
times corresponding to the assumed pattern of obser- 
vations. 

OSSE’s of this type were conducted by Charney 
et al. (1969), Halem and Jastrow (1970), Jastrow 
and Halem (1970, 1973), Williamson and Kasahara 
(1971), Kasahara (1972), Gordon et ai (1972), and 
others in preparation for the Global Weather Experi- 
ment. These studies provided an analysis of the Global 
Atmospheric Research Program (GARP) data require- 
ments, the "useful” range of predictability, and the 
need for reference level data. From the results, it 
was concluded that the assimilation of satellite-derived 
temperature profiles meeting the GARP data specifica- 
tions would yield a substantial improvement to the ac- 
curacy of numerical weather forecasts. A later study by 
Cane et a/.(1981) using a modifed OSSE procedure in- 
dicated similar potential for satellite surface wind data. 

An examination of the underlying rationale for the 
early simulation studies (Jastrow and Halem, 1973), 
as well as a comparison of the results of the above 
studies with the results of subsequent real data im- 


pact tests ( e.g . Halem et al, 1982; Baker et al . , 1984) 
indicated several important limitations. The most im- 
portant weakness stems from rhe fact that the same 
numerical model was used botn to generate the sim- 
ulated observations and to test the effectiveness of 
these observations. (This is referred to as the “iden- 
tical twin” problem.) Other weaknesses relate to the 
model-dependence of the studies and the specification 
of observational errors as random. These weaknesses 
can lead to overly optimistic results and incorrect con- 
clusions from an OSSE. 

The current methodology used for OSSE's was de- 
signed to increase the realism and usefulness of such 
experiments (Atlas et al . , 1985b). In essence, the anal- 
ysis/forecast simulation system consists of the follow- 
ing elements : 

1. A long atmospheric model integration using a very 
high resolution “state of the art 1 ' numerical model 
to provide a complete record of the assumed 
“true 1 state of the atmosphere (referred to as the 
“nature run" or “reference atmosphere"). For the 
OSSE to be meaningful, it is essential that the 
nature run be realistic, i.t. possess a model cli- 
matology, average storm tracks, etc. that agrees 
with observations to within prespecified limits. 

2. Simulated conventional and space-based observa- 
tions from the nature run. All of the observations 
should be simulated with observed (or expected) 
coverages, resolution, and accuracy. In addition, 
bias and horizontal and ve tical correlations of er- 
rors with each other and with the synoptic sit- 
uation should be introduced appropriately. Two 
approaches have been used for this purpose (At- 
las et al, 1985b; Hoffman et al, 1990). The 
simpler approach is to interpolate the nature run 
values to the observation ocations and then add 
appropriate errors. The more complicated (and 
expensive) approach is to attempt to retrieve ob- 
servations from the nature run in the same way 
as observations are retrieved from the real atmo- 
sphere. 

3. Control and experimental data assimilation cycles. 
These are identical to the assimilation cycles in an 
OSE except that only simulated data are assimi- 
lated. In order to avoid the identical twin problem, 
a different model from that used to generate the 
nature run is used for assimilation and forecast- 
ing. Typically this model has less accuracy and 
resolution than the nature model. Ideally, the dif- 
ferences between the assim ilation and nature mod- 
els should approximate thj differences between a 
“state of the art” model a id the real atmosphere. 


4. Forecasts produced from the Control and Experi- 
mental assimilations. As with the OSE's, forecasts 
are generated every few days to develop an inde- 
pendent sample. The analyses and forecasts are 
then verified against the nature run to obtain a 
quantitative estimate of the impact of proposed 
observing systems and the expected accuracies of 
the analysis and forecast products that incorporate 
the new data. 

An important component of the OSSE that im- 
proves the interpretation of results is validation against 
a corresponding OSE. In this regard, the accuracy of 
analyses and forecasts and the impact of already exist- 
ing observing systems in simulation is compared with 
the corresponding accuracies and data impacts in the 
real world. Ideally, both the simulated and real results 
should be similar. Under these conditions, no calibra- 
tion is necessary and the OSSE results may be inter- 
preted directly. If this is not the case, then calibration 
of the OSSE results can be attempted by determining 
the constant of proportionality between the OSE and 
OSSE impact as described by Hoffman et al. (1990). 

3. RESULTS AND CONCLUSIONS 

A large number of observing system experiments 
(OSE's) have been conducted to assess the impact of 
different types of atmospheric observations. In general, 
these experiments have shown space-based observing 
systems to be very important components of the Global 
Observing System. Nevertheless, conventional surface- 
based observations are still of critical importance. 

Observing system simulation experiments 
(OSSE’s) provide an effective means to evaluate the 
potential impact of a proposed observing system, as 
well as to determine tradeoffs in their design, and to 
evaluate data assimilation methodology. Great care 
must be taken to ensure realism of the OSSE’s, and 
in the interpretation of OSSE results. OSSE's have 
shown considerable potential for space-based lidar 
wind profiles and for improved space-based surface 
winds, however, quantitative results from OSSE's may 
only be valid for instruments meeting the expected 
accuracies and coverages that were simulated, and for 
the data assikilation systems used in the particular 
OSSE. 

At the conference, results will be presented from 
OSE's and OSSE’s that have been conducted over the 
last several years at the DAO. These will include OSE’s 
to assess the impact of current satellite observations 
in the DAO’s data assimilation system, and OSSE’s 
aimed at assessing the future impact of space-based 
lidar winds and other advanced sounders, as well as 



experiments to evaluate trade-offs in the design of fu- 
ture observing systems 
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